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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 . 1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

Responsive to communication(s) filed on 21 January 2005 . 
2a)[3 This action is FINAL. 2b)Q This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-30 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. , 

5) D Claim(s) is/are allowed. 

6) 03 Claim(s) 1-30 is/are rejected. 

7) Q Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)Q The drawing(s) filed on is/are: a)Q accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 !)□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



4) D Interview Summary (PTO-413) 
Paper No(s)/Mail Date. . 

5) □ Notice of Informal Patent Application (PTO-152) 

6) □ Other: . 

U.S. Patent and Trademark Office 

PTOL-326 (Rev. 1-04) Office Action Summary Part of Paper No./Mail Date 20050419 



Attachment(s) 

1) ^Notice of References Cited (PTO-892) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) □ Information Disclosure Statement(s) (PTO-1449 or PTO/SB/08) 

Paper No(s)/Mail Date . 
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DETAILED ACTION 



1. This action is responsive to the request for reconsideration filed on 1/21/2005. 
This action is made Final. 

2. Claims 1-30 are pending in the case. Claims 1, 1 1, and 21 are independent claims. 

Priority 

3. Acknowledgment is made of applicant's claim for foreign priority under 35 

U.S.C. 1 19(a)-(d), and based on application # 90 122 705 filed in Taiwan on 9/1 3/200 1 , which 
papers have been placed of record in the file. 



Drawings 

4. The drawings filed on 3/12/2002 have been approved by the Examiner. 



Claim Rejections - 35 USC §103 
5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art 
are such that the subject matter as a whole would have been obvious at the time the invention was made to 
a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be 
negatived by the manner in which the invention was made. 
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6. Claims 1-4, 6-14, 16-24, and 26-30 remain rejected under 35 U.S.C. 103(a) as being 
unpatentable over Myers et al, hereinafter Myers, (2002/0037104, 3/28/2002, provisional 
application filed on 9/22/2000). 

Regarding independent claim 1, Myers teaches the accepting, and capture of video with 
an image capture sensor— video-receiving module for receiving video source data, and decoding 
module (0033-0034, fig.l). 

Moreover, Myers teaches the detection, and location of text regions within capture video 
imagery. The text is detected according to text recognition device or computer-extracting the 
text part from the video data according to a production guide (0035-0036, fig.l). 

Furthermore, Myers fails to explicitly teach an illustration-extracting module for 
extracting a key frame from the video data according to the production guide and producing the 
book according to the extracted text part and illustration part. However, it would have been 
obvious to one of ordinary skill in the art at the time of the invention to have extracted an 
image — illustration part- from a video and index the image to produce an indexed photo album 
or book, using the extracted text, because Myers teaches using the extracted text to index images 
(0030, lines 1 1-29), thus, providing the benefit of organizing images, and text extracted from 
video, in order to quickly, and efficiently access those images. 

Regarding claim 2, which depends on claim 1, Myers teaches storing the extracted text 
for later processing with a word processor (0054, 0057). Myers fails to explicitly teach receiving 
a command from a user to edit contents of the book after the book is produced. However, it 
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would have been obvious to one of ordinary skill in the art at the time of the invention to have 
received a command with a word processor for editing the stored book, because Myers teaches 
above the later processing using the extracted text, thus, providing the flexibility, and 
convenience of a word processing program to edit the extracted text after words. 

Regarding claim 3, which depends on claim 1, Myers teaches a user storing the extracted 
text for later processing with a word processor — producing the book using a template stored 
(0054, 0057). 

Regarding claim 4, which depends on claim 1, Myers teaches a user deploying a 
notebook— production guide- for capturing video for extracting, and storing the extracted text 
for later processing with a word processor (0054, 0057). Myers fails to explicitly teach receiving 
a command from a user to select the production guide. However, it would have been obvious to 
one of ordinary skill in the art at the time of the invention to have received a command with a 
word processor for editing the stored book, because Myers teaches above the later processing 
using the extracted text, thus, providing the flexibility, and convenience of a word processing 
program to edit the extracted text after words. 

Regarding claim 6, which depends on claim 1, Myers fails to explicitly teach a caption- 
analyzing algorithm by which caption data and video data in the video data are analyzed, the 
text-extracting module extracts the caption data to obtain the text part according to the caption- 
analyzing algorithm, and the illustration-extracting module extracts image data from the video 
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data corresponding to the caption data as the illustration part. However, it would have been 
obvious to one of ordinary skill in the art at the time of the invention to have extracted an image 
from a video and index the image to produce an indexed photo album or book, using the 
extracted text, because Myers teaches using the extracted text to index images (0030, lines 11- 
29), thus, providing the benefit of organizing images, and text extracted from video, in order to 
quickly, and efficiently access those images. 

Regarding claim 7, which depends on claim 1, Myers teaches the capture of an image 
used for text detection — video data are analyzed according to an image sample (0034). Myers 
fails to explicitly teach the illustration-extracting module extracts image data to obtain the 
illustration part according to the image-analyzing algorithm, and the text-extracting module 
extracts the caption data to obtain the text part according to from the video corresponding to the 
image data. However, it would have been obvious to one of ordinary skill in the art at the time of 
the invention to have extracted an image from a video and index the image to produce an 
indexed photo album or book, using the extracted text, because Myers teaches using the 
extracted text to index images (0030, lines 1 1-29), thus, providing the benefit of organizing 
images, and text extracted from video, in order to quickly, and efficiently access those images. 

Regarding claim 8, which depends on claim 1, Myers teaches the capture of an image — 
according to an object- used for text detection (0034). Myers fails to explicitly teach the 
illustration-extracting module extracts image data to obtain the illustration part according to the 
image-analyzing algorithm, and the text-extracting module extracts the caption data to obtain 
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the text part according to from the video corresponding to the image data. However, it would 
have been obvious to one of ordinary skill in the art at the time of the invention to have extracted 
an image — illustration part- from a video and index the image to produce an indexed photo 
album or book, using the extracted text, because Myers teaches using the extracted text to index 
images (0030, lines 1 1-29), thus, providing the benefit of organizing images, and text extracted 
from video, in order to quickly, and efficiently access those images. 

Regarding claim 9, which depends on claim 1, Myers teaches the capture of an image 
used for text detection — text-extracting module extracts captions in the image data as the text 
part- (0030, 0034). Myers fails to explicitly teach the illustration-extracting module extracts 
image data as the illustration part. However, it would have been obvious to one of ordinary skill 
in the art at the time of the invention to have extracted an image — illustration part- from a video 
and index the image to produce an indexed photo album or book, using the extracted text, 
because Myers teaches using the extracted text to index images (0030, lines 1 1-29), thus, 
providing the benefit of organizing images, and text extracted from video, in order to quickly, 
and efficiently access those images. 

Regarding claim 10, which depends on claim 1, Myers teaches the capture of an image 
used for text detection by agglomerating the text recognition results from several video frames to 
obtain the best results from each frame and form a single result — scene/shots shifts of image 
data are analyzed- (0030, 0041). Myers fails to explicitly teach use the scene/shot shift 
analyzing algorithm as a selection and segmentation guide for the text and illustration part. 
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However, it would have been obvious to one of ordinary skill in the art at the time of the 
invention to have extracted an image — illustration part- from a video and index the image using 
the extracted text, to produce an indexed photo album or book, using the extracted text, because 
Myers teaches using the extracted text to index images (0030, lines 1 1-29), thus, providing the 
benefit of organizing images, and text extracted from video, in order to quickly, and efficiently 
access those images. 

Claims 11-14, 16-20 are directed towards a method for implementing the system found in 
claims 1-4, and 6-10 respectively, and therefore are similarly rejected. 

Claims 21-24, and 26-30 are directed towards a computer program product on a 
computer-readable medium for storing the steps found in claims 1-4, and 26-30 respectively, and 
therefore are similarly rejected. 

7. Claims 5, 15, and 25 remain rejected under 35 U.S.C. 103(a) as being unpatentable over 
Myers in view of Video Manga: generating semantically meaningful video summaries Uchihashi 
et al, October 1999, Proceedings of the seventh ACM international conference on Multimedia 
(Part 1), p.383-392, hereinafter Uchihashi. 

Regarding claim 5, which depends on claim 1, Myers teaches the capture of an image 
used for text detection (0034). Myers fails to explicitly teach extracts the audio data to obtain 
the text part However, Uchihashi teaches using speech recognition to extract text from video 
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(sect. 2, 4 th parag.). It would have been obvious to one of ordinary skill in the art at the time of 
the invention to have extracted an the text from audio found in the video, and extracted image 
from a video and index the image to produce an indexed photo album or book, using the 
extracted text, because Myers teaches using the extracted text to index images (0030, lines 11- 
29), and Uchihashi discloses capturing compact and visually pleasing summary capturing 
semantically important event (abst, sect. 6.2), thus, providing the benefit of organizing images, 
and text extracted from video, in order to quickly, and efficiently access those images. 

Claim 15 is directed towards a method for implementing the system found in claims 5, 
and therefore is similarly rejected. 

Claim 25 is directed towards a computer program product on a computer-readable 
medium for storing the steps found in claim 5, and therefore is similarly rejected. 

Response to Arguments 
8. Applicant's arguments filed 1/21/2005 have been fully considered but they are not 
persuasive. Applicant indicates that Myers does not disclose or suggest the production of a book 
using extracted text and video frame (pages 3-4). The Examiner disagrees, because Myers 
teaches the extraction of text from a video, and the indexing of images using the extracted text 
(0030, 0036). It would have been obvious to one of ordinary skill in the art to have indexed video 
images extracted from the video using the extracted text, because Myers teaches the indexing or 
cataloging of images found in a video (0030, lines 1 1-29, 0006). This would provide the benefit 
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of producing catalogs or books identifying the images found in the video using the extracted text, 
and permitting the user to quickly, and efficiently find and view the images found in the video, 
using the extracted textual identification. 

The video and image indexing is performed for various purposes, such as storage, and 
identification, and to increase marketing programs accuracy for displaying ads to a user (0078). 
The indexing is not just to increase marketing accuracy, but also for storage and identification of 
the video images, thereby providing a very strong motivation for producing photo albums or 
indexed catalogs of images. 

In response to applicant's argument that the examiner's conclusion of obviousness is 
based upon improper hindsight reasoning (page 5, parag.l), it must be recognized that any 
judgment on obviousness is in a sense necessarily a reconstruction based upon hindsight 
reasoning. But so long as it takes into account only knowledge which was within the level of 
ordinary skill at the time the claimed invention was made, and does not include knowledge 
gleaned only from the applicant's disclosure, such a reconstruction is proper. See In re 
McLaughlin, 443 F.2d 1392, 170 USPQ 209 (CCPA 1971). 

Therefore, in light of the statements above, claims 1-30 remain rejected. 

Conclusion 

9. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 . 1 36(a). 
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A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

L The prior art made of record and not relied upon is considered pertinent to applicants 
disclosure. Squilla et al. (Pat. # 6,288,719 Bl, AND 6,362,900 Bl), Savitsky et al. (Pat. # 
6,571,271 Bl), and Anderson (Pat. # 6,499,016 Bl). 

n. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Cesar B. Paula whose telephone number is (571) 272-4128. The 
examiner can normally be reached on Monday through Friday from 8:00 a.m. to 4:00 p.m. 
(EST). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 
Stephen Hong, can be reached on (571) 272-4124. However, in such a case, please allow at least 
one business day. 

Any response to this Action should be mailed to: 

Commissioner for Patents 
P.O. Box 1450 
Alexandria, VA 22313-1450 
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Or faxed to: 

• (703) 703-872-9306, (for all Formal communications intended for entry) 




CESAR PAULA 
PRIMARY EXAMINER 



4/19/05 



